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Abstract: 

This  presentation  describes  an  embedded  intelligent  agent  architecture  for  distributing  the  real¬ 
time  allocation  and  management  of  resources  throughout  networked  systems.  The  solution  is 
general  purpose,  and  has  been  developed  for  several  applications,  including  shared  electronic 
systems  and  spectrum  allocation.  The  agent  architecture  was  derived  from  a  combination  of 
prior  agent  and  resource  management  developments  from  Lockheed  Martin  ATL. 

To  guide  development  and  evaluate  the  distributed  agent  concept  prior  to  detailed 
implementations,  simulation  models  were  constructed.  A  successful  agent  architecture  was 
derived  which  relies  on  agent  to  agent  negotiation  for  resolving  allocation  decisions  in  a 
distributed  and  parallel  context.  The  simulations  led  to  new  methods  for  developing  agents 
having  sufficiently  small  program  code  and  data  structures  consistent  with  embeddable 
implementations.  Further  developments  led  to  highly  scalable  computing  solutions. 

On  DARPA's  Next  Generation  Communications  (XG-Comms)  project,  the  agent  technology  was 
studied  to  leverage  Software  Defined  Radios  (SDR)  and  related  emerging  wireless  systems  of  the 
future,  for  improving  total  spectrum  utilization,  especially  for  the  continuously  changing  heavy 
traffic  on  the  battlefield. 

The  XG-Comms  program  goals  require  re-using  spectrum  in  time,  space,  frequency,  and 
modulation,  while  minimizing  interference  to  adjacent  systems.  The  real-time  distributed  nature 
of  the  challenge  was  well  suited  to  agent  solutions.  Results  from  the  simulations  of  agents 
allocating  spectrum  will  be  shown.  The  simulations  showed  that  the  unique  developments  in  the 
agent  design  enable  the  embedded  agents  to  increase  spectrum  usage  by  more  than  lOx,  while 
consuming  less  than  0.2%  spectral  overhead  for  control  and  negotiation  under  very  high 
demanded  loads. 
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Challenge:  Embedded  Real-  Time  Management  of 

Distributed  Resources 

-  General  reoccurring  problem.  Resources  may  be  heterogeneous 

-  NP  Hard  -  Many  possible  solutions;  Most  not  sufficient. 

Example  Applications: 

-  Mission  scheduling,  planning,  logistics,  multi-  sortie  control 
of  autonomous  units,  teams  of  UAV's,  UGV's,  UWV's, .... 

-  Multi-  computer  task  scheduling,  load  leveling. 

-  Plant  management,  power  &  operations  scheduling, 
vehicle  signature  control. 

-  Network  load  balancing,  routing,  wireless  spectrum  allocation. 

-  Carrier  deck  operations  scheduling. 

-  Traffic  flow  management,  optimization,  intelligent  highways. 

-  Management  ==>  Continuous  re-  planning. 
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Existing  Solutions: 

-  Bin  Packing  (Coffman  1998,  et.  al.) 

-  First- Fit  (J.  Ullman,  1973) 

-  Cookie-  Cutter  (Hoffman  1998) 

-  HARMONICM  (JL  Yowell,  1999) 

-  Multidimensional  Bin  Packing  Algorithms  (Kou/Markowsky  'll) 

Tendency:  One  shot,  centralized,  static,  pre-  schedule  off-  line. 

Not  intended  for  continuous  real-  time  operations. 

Traditional  First  Fit  Scheduling  Algorithm 
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Approach: 

-  Leverage  previous  methods  by  distributing  and  extending  them. 

1.  Select  distribution  framework:  Intelligent  Agents 

2.  Extend  allocation  algorithms  within  agent  paradigm. 

Intelligent  Agent: 

-  Independent  software  process  w/Persistence,  Continual  operation. 

-  Has  perception,  goals  and  logic  to  achieve  goals.  Intent  oriented  method. 

-  Serves  as  proxy  for  an  application  stake-  holder. 

-  Collectively  finds  global  solution  by  cooperative  negotiations  w/other  agents. 

-  Light  weight,  embeddable,  mobile. 

Agent  Method 
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Agent  &  Resource  Simulation  Environment 

-  Complex  systems  -  -  Difficult  to  understand. 

-  Exploit  modeling  &  simulation  for  rapid  efficient  exploration  &  development. 

-  Must  investigate  Temporal,  Spacial,  and  Functional  aspects: 

-  Temporal  =  When  resource  needed  or  used. 

-  Spatial  =  Where  requested  /  consumed. 

-  Functional  =  Express  complex 


agent  allocation  rules. 

-  Used  ATL  CSIM 

-  www.atl.lmco.com/proj/csim 

-  Good  agent  simulator. 

-  Multi-  domains/levels 
with  common  simulator. 

-  Provides  mission-  level  context 
for  engineering  models  to 
assess  mission  success  impacts. 


Time 


(Architecture)  Functionality 
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Experiments: 
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Metrics: 

-  Response  Time  (min,  mean,  max,  variance) 

-  Overhead,  agent  communications  (bytes/sec) 

-  Scalability  (growth  as  function  of  complexity) 


Simulation  Approach: 


6 


Agent  Design 

-  Tried  several  approaches 

-  Key  discovery  -  -  > 

Agents  with  their  own 
internal  world  models 
reduce  inter-  agent 
negotiations. 

-  World  models  enable 
agents  to  test  tentative 
future  actions  a  priori. 

-  Enables  continuous 
predictive  re-  planning, 

-  -  >  Anticipatory 

optimal  sequencing. 
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-  Dual  mode  operation: 

1 .  Quick  reaction  subsystem, 

2.  Background  optimization  subsystem. 

-  Agents  maintain  own  world  models  through 
observation  of  environment/past  experience. 

-  Agent  considers  decision  alternatives  by 
simulating  outcome  on  internal  model. 

-  Requires  lightweight,  portable,  embeddable 
simulator  such  as  CSIM. 

-  During  simulations  in  CSIM,  agents  launch 
their  own  mini-  simulations  within  virtual  world. 


Agent 

Background 

Optimization 

Subsystem 


Quick 

Reaction 

Subsystem 


Physical 

Plant 
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Application  Example  1 
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Agents  Applied  to  Real-  Time  Software  Task  Scheduler 

Agent  based  controller 

Oynvnlc  SCHEDULER  for 
Htrttvvo  /  Software  SmiLiJtlon 


Software  OF  G  SCHE  DUE  I  R 


Hardware  Are h He c tu re 


Task 

Management 

Agents 


Execution 

Management 

Agents 


Physical  Hardware 
System 
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Application  Example  1 
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Simulation  Process 
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Scheduling 

Latency 


Application  Example  1 
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Application  Results: 

-  Combination  of  advanced  algorithms  found  to  enable  efficient 
distributed  operation. 

-  Neither  agent  paradigm  by  itself,  nor  advanced  algorithms  alone 
could  account  for  improvement  individually,  but  only  in  combination. 

-  Greater  scalability  indicated  for  new  approach  on  applied  scenarios. 
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Next  Generation  Communications  -  (XG  Comms) 


-  DARPA  program  to  demonstrate  lOx  >  usable  wireless  spectrum. 

-  Spectrum  is  presently  allocated  statically,  centrally,  but  not  efficiently. 

-  All  spectrum  is  allocated.  Little  occupied  at  any  one  time  or  place. 

-  Is  multi-  dimensional  (Time,  Freq,  Area,  Modul)  dynamic  allocation  app. 
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Spectrum  Management 

-  CSIM  Simulations  at 
multiple  simultaneous  levels. 

-  Enable  agent  interaction  in 
realistic  mission  scenarios. 

-  Multi-  view  visualizations 
aid  understanding. 


-  Agent  models  under  each 
vehicle  model,  contain 
agent  submodels. 
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Yar  Scops  1  -  Global  to  all. 
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Options 


-  Multi-  dimensional  view  shows 
allocations  in  time  duration 
(right-  left)  vs.  location  (depth) 
vs  frequency  (vertical)  and 
vs  priority  (color/owner). 


Spectrum  Management 

-  World  models  of  agents 
within  each  radio  view 
show  allocations  vs.  time 
(horiz.)  and  vs.  freq. 
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Conclusions 

General  purpose  distributed  real-  time  control  method. 
Advantages: 

-  Decentralized,  continuous  operations,  real-  time. 

-  Embeddable,  lightweight. 

-  Minimizes  central-  dispatch  communications  . 

-  Improves  efficiency,  reduces  disruptions  &  response  delays. 

-  Greater  scalability  than  centralized  solutions. 

Risks: 

-  New  inter-  agent  communication  overhead. 

-  Convergence,  solution  quality,  stability,  limit  cycles,  chaos. 

-  Predictability,  ability  to  set  bounds. 

-  The  unknown. 

Lessons  learned: 

-  Modeling  &  simulation  essential  for  agent  deployment. 
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